Equality statements for entropy change in open systems 
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Abstract 

The entropy change of a (non-equilibrium) Markovian ensemble is calculated from (1) the ensem- 
ble phase density p(t) evolved as iterative map, p(t) = M.(t)p(t — At) under detail balanced transition 
matrix M(<), and (2) the invariant phase density ir(t) = M(t)°°7r(t). A virtual measurement pro- 
tocol is employed, where variational entropy is zero, generating exact expressions for irreversible 
entropy change in terms of the Jeffreys measure, Jit) — YlrlPtt) ~ 7r (^)] m [p(0/ 7r (0]) an d f° r re ~ 
versible entropy change in terms of the Kullbach-Leibler measure, 2?ifi(t) = Ylr 77 (®) m [ 7r (^)/ 7r (^)]- 
Five properties of J are discussed, and Clausius' theorem is derived. 
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Reversible manipulation is the principal tool of the thermodynamicist. Reversibil- 
ity appears in two forms: the quasi-static time-forward reversible transition and the 
microscopically-reversible time-reversed (or adjoint) stochastic transition Ap- 
plication of microscopic reversibility to the path integral formulation of stochastic pro- 
cesses jj, Q has resulted in a set of fluctuation theorems (FT) for systems arbitrarily far 
from equilibrium 6]. FT, despite their elegance, do not provide a much needed general 
definition of entropy change — an equality statement providing the entropy change for any 
transition — of an ensemble of Markovian systems. Here, using both the quasi-static and 
time-reversed transitions in the path integral approach to the dynamics of a Markovian 
system, we produce these equality expressions for microscopic and macroscopic entropy 
change. 

A collection of M classical particles undergoing Hamiltonian dynamics is partitioned, 
through scale separation, into system and bath 7, ^j]. The system, consisting of N par- 
ticles is transformed into a Markovian stochastic process described by 6iV generalized 
coordinates. Phase space and time are taken as discrete quantities. Each coordinate is an 
m-tuple, and time consists of equally spaced intervals, At = tj+i — Tj. The system tra- 
jectory is given as the time evolution of a phase point, <t(t) = 5 [(x; r) — (xq; t)], in phase 
space r, a (6N x m)-tuple, with x, xq S T, according to the stochastic iterative map 

a{n) = M^fo-i), (1) 

n 

where M. n = M Ti (<r(T$)|er(Ti_i)) is interpreted as a stochastic matrix [9(. Real systems, 
which operate under colored (OU) noise, are managed by requiring that the discrete time 
step in (HI) be much longer than the correlation time of the noise. Following Gibbs, we 
consider an ensemble of such collections. Interpreting M. Ti as a transition matrix (rather 
than a stochastic matrix) [10] and defining the phase probability as the normalized density 



of phase points, P (rj) = cr(rj), the dynamics of the ensemble is a time-inhomogenious 
Markov chain 

P{n)=M n P{ n - 1 ). (2) 
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FIG. 1: Ensemble undergoing a general perturbation. Evolution of phase density p along path 7, 
consisting of small displacements 7$. The entropy change under 7, is measured along the virtual 
paths 7°, 7 2 b , 7j C involving the invariant density 7r. 



It is assumed that M Ti is a known quantity obtained through experimental parametrization 
or on the basis of theory. From M T . and given starting phase density p(tq) = P(tq), two 
time-dependent quantities of interest are determined — the time-dependent phase densitiy 
p(ri) and the time-dependent invariant phase density, 7r(rj) = M^°7r(rj), obtained as follows: 
the dynamics at time Tj are stoppped, then the density is evolved in virtual time ij — > 00 
under stationary M r . , according to (J2J) - For nonequilibrium system ensembles, ir is a 
virtual quantity. For equilibrium ensembles that undergo quasi-static perturbation, tt is a 
real quantity. Our results apply to ensembles that evolve according to ([2]) with transition 
matrices M T . that are Hermitian. These systems possess three important properties [11]. (i) 
Microscopic reversibility — M Ti is self- adjoint J], 4]: M Ti 7r(Tj) = M r .7r(rj). (ii) Invariance — 
the invariant distribution of a stationary Markov process is independent of the ensemble 



history: lim^ 



I* n p(rj) = 7r(rj). (in) Stationarity — with invariant density: tt(t, 



M T4 7r(rj). Microscopic reversibility is a property of physical systems fl^ ] and a fundamental 
postulate of physics [2]. 

General Perterbation — With these properties in mind, we calculate the entropy change 
of a thermodynamic ensemble of Markovian systems undergoing an arbitrary forced per- 
turbation. Using the prescription in Fig. [lj entropy change for the ensemble transition 
along the path increment 7^ (bold arrows) is evaluated using three measurements that, 
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being virtual, do not perturb the system . The path 7 taken by one system start- 
ing at 0"(io) is given by the time-ordered collection of phase points visited by the system 
{a(t )a(ti) ■ ■ ■ a(t n )} under evolution by M 4o M tl • • • M tn , 

7 = a(t ) — ► (X(ti) — > ■ ■ ■ a(t n -i) — ► <j(t n ). 

The adjoint path is 

~_~ t . , M tn Mtj, ^, M tl 

where Mf. = M+. (or(tj)|or(^-_i)). The adjoint path starts where 7 ends, a (to) = a(t n ). For a 
system in state cr(to), the probability that it follows the path 7 is given by the product of the 
single time step transition probabilites, V(j\p) = YYi=i^-Ui where p = P(a(to)). Similarly, 
for the adjoint transition, the conditional adjoint path probability is V(i\p) = IliLzt^**' 
where p = P(a(to)) is the probability that the system starts in state a (to)- Using the 
definition of conditional probability and taking the quotient of path probabilities, we obtain 

V(l) = V{l\v)V = tt M u p 

p(rf) v(ci\p)p 11 m u p K ) 

The conditional path probability is also given as a function of action [l|, Q], V^lp) = 
exp(— .4^(7)), and we note the correspondence, lnM^ = —A^. Defining the micro- 



scopic entropy change of the collection 



„ 
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5S = InVh) -InVR) 

r ~ 1 , (4) 

= In [p/p] e E>M ti -lnM ti 

we obtain a microscopic entropy balance equation 

5S = A5 7 + In [p/p\ , (5) 

involving gain of entropy by the heat bath, A5 7 = Yli hiM^/M^, and gain of entropy by 
the system, ln(p/p). In entropy and action are on equal footing: entropy is proportional 
to the logarithm of exponentiated action. Evaluation of ([5]) is straightforward when some 
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path 7 can be identified where 5S = 0. We show that the measurement in Fig. CD is along 
such a path. 

Each path 7? involves the evolution in virtual time to, t\, . . . , t n of the ensemble start- 
ing in p(rj_i) to the stationary distibution 7r(rj_i). The evolution is a virtual time- 
homogeneous transition governed by the transition matrix M Ti l . For each 7?, ([5]) provides, 
A«S 7 ? = SSrya — In \p(ri-i) /n^Ti-i)]. Using the property of invariance (ii), the path proba- 
bility density is 

V( 7 ?)=M™_ i p(t 1 ^) = tt(t 1 _ 1 ). (6) 

Application of detailed balance (i) followed by sequential application of stationarity (Hi) 
to the adjoint path probability density yields 

= vr(ri_i). 

For each path 7?, the microscopic entropy change of the collection is zero, 6S^? = 
In [7^(7") /V (7")] = 0, yielding, A«S 7 ? = — In [p(Tj_i)/7r(Tj_i)]. The macroscopic entropy 
over the disjoint paths 7? is the ensemble averaged entropy 

n-l 

AS 7 . = -X)<lBb(Ti)Mr i )]) p(7i) , (8) 

i=0 

where (/(T))^ = £r x(r)/(T). 

Each path 7^ involves the virtual evolution of the invariant starting distribution 
7r(rj_i) to the invariant distibution 7r(rj) under (virtual) time-homogeneous evolution by 
M Ti . For each 7., (|5|) provides, AS b = 5S t — In [7r(rj_i)/7r(rj)]. By the same argu- 
ments used in ([6]) and ((7J), for each path 7?, <5<5> 7 i, = In )/V(^\)\ = 0, yielding, 
AS b = — In [7r(Tj_i)/7r(Tj)]. We concatenate the 7^ path segments into a continuous vir- 
tual path 7 6 for the evolution of it. The microscopic entropy over the thermodynamically 
reversible path -y b is, after cancelling terms, AiSy, = Yli=l ^^7? = ~~ ^ n [ 7r ( T o)/ 7r ( T n)]- The 
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reversible macroscopic entropy flow into the system during 7 b is 

AS re „ = -AS 7 „ = (In [vr(ro)/vr(r n )]), {To) , (9) 



which is the relative, or Kullbach-Leibler, entropy Vkl[x, y] = ^ r xln(x/y) [17I ]. 

Each path 7? involves the adjoint (virtual) time-homogeneous evolution of the ensemble 
starting from the invariant distribution 7r(rj) to the real distribution p(rj) under M Ti . For 
each 7? the entropy © is, AsS 7 ? = SS^c — In [7r(rj)/p(ri)]. Again, by the same arguments 
used in Q and ©, for each path 7?, &S 7 ? = In \P{ n /f)/V(^)i )] = 0, yielding, A<S 7 ? = 
— In [ft(Ti)/p(Ti)]. The macroscopic entropy change over the disjoint paths 7? is 

n 

Aa y c = -X)(lii[7r(r i )Mri)]) w(Ti) . (10) 

i=l 

From ([8]) and (|10p . the irreversible macroscopic entropy flow into the system over 7" and 
7 C , ASj rret , = -(AS 7 « + AS 7 c), is 

n-l 

AS irrei) = £ + ^ (In Wi)/<n)]) p{Ti) - <n) + #n, (11) 
i=i 

where 5 = ^ r p(r ) lnp(T )/7r(T ) and B n = ^ r Tr(T n )\nir(T n )/p(T n ) are boundary 
Kullbach-Leibler integrals. The sum in (jlip is over the Jeffreys invariant divergence mea- 
sure, J7"[x, y] = ^ r (x — y)ln(x/y) 18]. Jeffreys and others have commented on 



the many remarkable properties of J . 

Perturbation #1 — We apply the results obtained for the general perturbation to two 
specific, and important, perturbations (Fig. [2]). In Fig. [2h., the system is perturbed 
from one equilibrium state to another. For the (virtual) equilibrium path (dashed line), 
macroscopic entropy flow into the system is, from Q, the Kullbach-Leibler entropy, 
AS rei , = T>kl[^(to), 7r(r n )]. For the (real) non-equilibrium path (solid line), macro- 
scopic entropy is the sum of the reversible entropy ([9]) and the irreversible entropy (jlip . 
AStot = AS re „ + AS irre „. Here, p(r ) = 7r(r ) and p(r n ) = vr(r n ), causing B and B n to 
vanish. The irreversible entropy flow into the system is the discrete time integral over the 
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(a) 7T(T n ) 




FIG. 2: Two perterbations. (a) Non-equilibriuum perturbation with p starting and ending in 
equilibrium (convergence of p and tt). (b) Periodic perturbation with closed orbits of p and n. 

Jeffreys invariant measure, ASi rrev = Y^i=i ^[p( t «)' ^l 7 *)]- The total entropy flow into the 
system is 

n-1 

AS tot = V KL [ir(T ),7r(T n )}+Y,J[p(Ti)^(Ti)}- (12) 

8=1 

Perturbation #2 — In Fig. [2)3, we consider an ensemble of systems undergoing periodic 
perturbation with period ujAt, where M Ti = ~M Ti+uj . Using (|2|), an initial phase density 
p( T o) = -P( (T (' r o))) is prepeared from some arbitrary phase density p(r_r>) through the 
equilibration process: P {t ) = (M Tw M Tw _ 1 . . . M T1 ) n/w p(r_ n ), where il/u £ 1+ > 1. While 
the Poincare recurrence time for any one system may be extremely long, the recurrence 
time for the ensemble is a; At. From ([9]) and using the property, tt(tq) = 7r(r n ), we obtain 
AS rei) = T>kl[^{jq), vr(r n )] = 0. From (fTTI) and the property, p(tq) = p(r n ), we obtain 
ASi rrev = Ya=i J'[p( T i)i 7r ( T «)]- The total entropy flow into the system over one cycle of 
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perturbation is 

n 

AS tot = Yj J Wi)Mn)]- (13) 

i=l 

The total entropy transferred from the bath to the system over a thermodynamic cycle is 
the time integral of the Jeffreys divergence between real and invariant phase densities. 

Properties of J — The properties of J generate some important conclusions. J is almost 
positive definite, meaning (i) i7[x, y] > and (ii) J^x, y] = only when x = y. (Hi) J 
is symmetric: c7[x, y] = JT^x]. (iv) J' is a linear measure: det (d 2 J / d~x.id~x.j) = 0. 
T>kl satisfies (i), (ii) [2o| and (iv). See EPAPS Document No. [] for proofs and further 
discussion. 

For an isothermal (cannonical) system, AS = —AStot, is the heat flow per temperature 
from the system to the bath during a periodic perturbation, AS = § (3dQ, where (3 = 
l/ksT is inverse temperature in units of energy. Clausius' statement of the second law of 
thermodynamics is obtained from (I13p and properties (i) and (ii), 

(3dQ < 0. 

Practical application of (fT3j) to many-body systems derives from a fifth property of J ', 
(v) decomposability [3]: for a system with a decomposable Markov transition matrix 

A 
B 

the phase density decomposes: p = paPb, ^ = t^a^b, phase space decomposes: T = T^r^, 
and the invariant measure decomposes: J[p, it] = J a \pa-, ka] + J B [pb,^b\- Defining, 
AS fc = Y^i ■Jt\Pi-> ^i]) an d using (v), we find that macroscopic entropy is extensive, AS = 
AS^+AS B . The most immediate application of decomposability is the overdamped system 
where Tb comprises momentum space and momentum is always equilibrated, ps = ttb. 
Using (|13p and properties (ii) and (v), we obtain, AS = AS^. Further reduction may be 



possible with a suitable choice of basis for Ta 
system 



21[ and also upon coarse graining of the 
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A. Some comments 

Aside from generating equality statements for reversible and irreversible entropy change, 
perhaps our most important contribution is providing a purely virtual integration protocol 
for evaluating a variation. Being virtual, the integration can not introduce uncertainty into 
the system [l|, l2| . This measurement protocol is likely to find application beyond statistical 
mechanics. 

We note that Gibbs entropy, S = — lnp, [3, 4] and Boltzmann entropy, S = — Inn, 
follow as boundary terms from the definition of variational entropy SS when p or p are 
evaluated at n. 

B. Some properties of the Jeffreys divergence J and the Kullback-Leibler diver- 
gence T> 

The Jeffreys divergence measure is defined 

J[x,y] =^(x-y)ln(x/y). 
r 

The Kullback-Leibler divergence measure is defined 

V[x,y] =^xln(x/y). 
r 

Theorem .1 J is almost positive definite, meaning (i) J~[x, y] > and (ii) J7[x, y] = 
only when x = y. 

Proof We consider, element-wise, the probability densities x = {x}, y = {y}, and the 
Jeffreys measure, J7"[x, y] = ^ r Ji[x, y]. For x > y, (x — y) > and ln(x/y) > 0; therefore, 
J7i[x,y] = (x — y)\n(x/y) > 0. For x < y, (x — y) < and ln(x/y) < 0; therefore, 
,Ji[x,y] > 0. For x = y, {x — y) = and ln(x/y) = 0; therefore, J~i[x,y] =0. | 
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Theorem .2 J is symmetric: i7[x, y] = J7[y,x] 



Proof J is evaluated element- wise. Both terms (x—y) and hx(x/y) are odd under exchange 
of x and y. The product of two odd functions is even. | 

Theorem .3 J does not satisfy the triangle inequality: J\x,z\ < J[x,y\ + J~[y,z]. 

Proof The proof is by example (Nikolai Chernov, personal communication). Let x = 
[0.25,0.75], y = [0.50,0.50], and z = [0.75,0.25]. J[x,y] = 0.27; J[y,z] = 0.27; J[x,z] = 
1.10. We obtain, J[x,z] > J[x,y] + J[y, z]. | 

By not satisfying the triangle equality, the Jeffreys measure falls short of being a topologic 
metric Q]. For this reason, the term "Jeffreys divergence measure" is the preferred over 
the "Jeffreys distance measure." 

Theorem .4 J is a linear measure. 

Proof The Hession of J~[x,y], 

^ ^ d 2 J/dxdx d 2 J/dxdy 

d 2 J/dydx d 2 J/dydy 

is evaluated: d 2 J/dxdx = (x + y)/x 2 , d 2 J/dydy = (x + y)/y 2 , and d 2 J/dxdy = 
d 2 J/dydx = —{x + y)/xy. By substitution we find, detH(J') = 0. | 

Theorem .5 T> is a linear measure. 

Proof The Hession of X>[x,y] is evaluated: d 2 T>/dxdx = 1/x, d 2 T>/dydy = x/y 2 , and 
d 2 V/dxdy = d 2 V/dydx = -l/y. By substitution we find, det H(V) = 0. | 
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